Genetic Algorithms with Mapreduce Runtimes

نویسندگان

  • Fei Teng
  • Doga Tuncay
چکیده

Data-intensive Computing has played a key role in processing vast volumes of data exploiting massive parallelism. Parallel computing frameworks have proven that terabytes of data can be routinely processed. Mapreduce is a parallel programming model and associated implementation founded by Google, which is one of the leading companies in IT. Genetic Algorithms have increasingly applied on parallel computing to large scale problems. Since, GAs have the parallelism in their nature, they can be easily applied on Parallel Runtimes such as MPI, Hadoop Runtime and Twister[9]. Our researches have shown that, Genetic Algorithms were very successful and can be modeled on MPI (Message Passing Interface) and Mapreduce Models. We will explain the nature of Genetic Algorithms, how they are efficient to use Parallel computing networks and how can they be applied MapReduce Model. In this project, we will be applying Genetic Algorithms, primarily the basic algorithm Onemax Problem on Hadoop Mapreduce Framework and the Twister Iterative Mapreduce Framework and do the performance Analysis. Our expectation from this project, to prove that the GAs will perform better performance on Iterative Model from its nature and Twister’s computing model.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Exploiting Spatio-Temporal Tradeoffs for Energy Efficient MapReduce in the Cloud

MapReduce is a distributed computing paradigm that is being widely used for building large-scale data processing applications like content indexing, data mining and log file analysis. Offered in the cloud, users can construct their own virtualized MapReduce clusters using virtual machines (VMs) managed by the cloud service provider. However, to maintain low costs for such cloud services, cloud ...

متن کامل

Cloudbreak: Accurate and Scalable Genomic Structural Variation Detection in the Cloud with MapReduce

The detection of genomic structural variations (SV) remains a difficult challenge in analyzing sequencing data, and the growing size and number of sequenced genomes have rendered SV detection a bona fide big data problem. MapReduce is a proven, scalable solution for distributed computing on huge data sets. We describe a conceptual framework for SV detection algorithms in MapReduce based on comp...

متن کامل

Cloud Computing Technology Algorithms Capabilities in Managing and Processing Big Data in Business Organizations: MapReduce, Hadoop, Parallel Programming

The objective of this study is to verify the importance of the capabilities of cloud computing services in managing and analyzing big data in business organizations because the rapid development in the use of information technology in general and network technology in particular, has led to the trend of many organizations to make their applications available for use via electronic platforms hos...

متن کامل

A Study of Skew in MapReduce Applications

This paper presents a study of skew — highly variable task runtimes — in MapReduce applications. We describe various causes and manifestations of skew as observed in real world Hadoop applications. Runtime task distributions from these applications demonstrate the presence and negative impact of skew on performance behavior. We discuss best practices recommended for avoiding such behavior and t...

متن کامل

A comparison on scalability for batch big data processing on Apache Spark and Apache Flink

*Correspondence: [email protected] 1Department of Computer Science and Artificial Intelligence, CITIC-UGR (Research Center on Information and Communications Technology), University of Granada, Calle Periodista Daniel Saucedo Aranda, 18071 Granada, Spain Full list of author information is available at the end of the article Abstract The large amounts of data have created a need for new fram...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2011